DTE AICCOMAS 2025

Adaptive Random Fourier Features Training Stabilized By Resampling With Applications in Image Regression

  • Kammonen, Aku (KAUST)
  • Pandey, Anamika (RWTH Aachen University)
  • von Schwerin, Erik (KAUST)
  • Tempone, Raúl (KAUST)

Please login to view abstract download link

ABSTRACT I will present an enhanced adaptive random Fourier features (ARFF) training algorithm for shallow neural networks, building upon the work introduced in [1]. This improved method, introduced in [2], uses a particle filter type resampling technique to stabilize the training process and reduce sensitivity to parameter choices. With resampling, the Metropolis test may also be omitted, reducing the number of hyperparameters and reducing the computational cost per iteration, compared to ARFF. I will show experiments demonstrating the efficacy of the proposed algorithm in function regression tasks, both as a standalone method and as a pre-training step before gradient-based optimization, here Adam. Furthermore, we apply our algorithm to a simple image regression problem, showcasing its utility in sampling frequencies for the random Fourier features (RFF) layer of coordinate-based multilayer perceptrons (MLPs). In this context, we use the proposed algorithm to sample the parameters of the RFF layer in an automated manner. REFERENCES [1] Kammonen et al., Adaptive Random Fourier Features with Metropolis Sampling. Foundations of Data Science, 2(3):309--332, 2020. [2] Kammonen et al., Adaptive Random Fourier Features Training Stabilized By Resampling With Applications in Image Regression. arXiv:2410.06399, 2024